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OBJECTIVE:  This  study  explored  the  effects  of  a  cognitive  task 
analysis  (CT A) -informed  curriculum  to  increase  surgical  skills 
performance  and  self-efficacy  beliefs  for  medical  students  and 
postgraduate  surgical  residents  learning  how  to  perform  an 
open  cricothyrotomy. 

METHODS:  Third  -year  medical  students  and  postgraduate 
year  2  and  3  surgery  residents  were  assigned  randomly  to  either 
the  CTA  group  (n  =  12)  or  the  control  group  (n  =  14).  The 
CTA  group  learned  the  open  cricothyrotomy  procedure  using 
the  CTA  curriculum.  The  control  group  received  the  tradi¬ 
tional  curriculum. 

RESULTS:  The  CTA  group  outperformed  the  control  group 
significantly  based  on  a  19-point  checklist  score  (CTA  mean 
score:  17.75,  standard  deviation  [SD]  =  2.34;  control  mean 
score:  15.14,  SD  =  2.48;  p  =  0.006).  The  CTA  group  also 
reported  significantly  higher  self-efficacy  scores  based  on  a  140- 
point  self- appraisal  inventory  (CTA  mean  score:  126.10,  SD  = 
16.90;  control:  110.67,  SD  =  16.8;  p  =  0.029). 

CONCLUSIONS:  The  CTA  curriculum  was  effective  in  in¬ 
creasing  the  performance  and  self-efficacy  scores  for  postgrad¬ 
uate  surgical  residents  and  medical  students  performing  an 
open  cricothyrotomy.  (J  Surg  68:403-407.  ©  201 1  Association 
of  Program  Directors  in  Surgery.  Published  by  Elsevier  Inc.  All 
rights  reserved.) 
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INTRODUCTION 

In  the  current  surgical  education  model,  mastery  in  surgery  is 
highly  dependent  on  the  ability  of  an  expert  surgeon  to  transfer 
the  necessary  knowledge  and  skills  to  novice  resident  learners. 
This  has  become  increasing  challenging  over  the  past  decade 
because  of  work-hour  restrictions,  patient  safety  concerns,  and 
decreased  operating  room  availability.  To  address  this  issue, 
many  programs  have  moved  surgical  skills  education  into  sim¬ 
ulated  environments  where  trainees  are  often  required  to  dem¬ 
onstrate  competency  before  performing  a  procedure  on  a  pa¬ 
tient.  Many  programs  now  rely  on  structured  checklists  and  a 
standardized  skill  curriculum.  A  significant  barrier  to  this  edu¬ 
cational  model  is  that  it  relies  on  subject  matter  experts  to  teach 
and  generate  instructional  materials.  Recent  studies  examining 
the  teaching  of  complex  knowledge  have  shown  that  experts 
omit  a  significant  amount  of  knowledge  when  trying  to  describe 
a  task,1"5  largely  because  repeated  years  of  practice  causes 
knowledge  to  become  automated;  automated  knowledge  is  no 
longer  accessible  to  conscious  processes. 4-6 

Cognitive  task  analysis  (CTA)  refers  to  a  variety  of  methods 
used  to  elicit  the  knowledge  and  skills  experts  use  to  solve  dif¬ 
ficult  problems  and  perform  complex  procedures.7  CTA  meth¬ 
ods  have  been  employed  effectively  to  design  surgical  skills 
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training  and  improve  performance  by  providing  step-by-step 
protocols  and  checklists  that  identify  expert  decisions  made 
during  a  procedure.8'9  CTA  is  a  viable  option  for  developing 
instruction  materials  because  it  allows  analysts  to  break  down 
the  complex  automated  skills  of  experts  into  manageable  tasks 
and  steps.7 

Another  consideration  for  educators  when  designing  surgical 
skills  curricula  is  the  influence  of  self-efficacy.  Self-efficacy  be¬ 
liefs  are  judgments  a  person  makes  about  his  or  her  ability  to 
perform  a  task  successfully  and  has  been  shown  to  effect  learner 
motivation  and  performance  achievement.10  In  academic 
settings,  previous  research  has  found  that  higher  self-efficacy 
is  related  to  greater  use  of  self-regulated  learning  strategies  as 
well  as  higher  achievement.  According  to  Bandura,1  ’ 
how  individuals  behave  can  often  be  predicted  better  by  their 
beliefs  about  their  capabilities  than  by  what  they  are  actually 
capable  of  accomplishing;  these  beliefs  help  determine  what 
individuals  do  with  the  knowledge  and  skills  that  they  have. 
Compared  with  persons  who  doubt  their  capabilities,  those 
with  high  self-efficacy  for  accomplishing  a  task  participate 
more  readily,  work  harder,  persist  longer  when  they  encoun¬ 
ter  difficulties,  and  achieve  at  a  higher  level.10'12  Competency- 
based  training,  such  as  CTA-informed  instruction,  has  been  shown 
to  influence  self-efficacy  beliefs  positively  in  other  disciplines10; 
however,  CTA  needs  further  exploration  in  surgery.  Although 
CTA-informed  curricula  has  had  positive  effects  on  surgical  resi¬ 
dent  and  medical  student  performance,  the  effects  of  CTA- 
informed  instruction  on  self-efficacy  have  yet  to  be  examined. 

The  purpose  of  this  study  was  to  determine  whether  CTA- 
informed  instruction  had  a  positive  effect  on  both  surgical  per¬ 
formance  and  self-efficacy  beliefs  of  postgraduate  surgical  resi¬ 
dents  and  medical  students  learning  how  to  perform  an  open 
cricothyrotomy. 

METHODS 

Cognitive  Task  Analysis 

Part  1:  task  list.  Six  expert  trauma  surgeons  participated  indi¬ 
vidually  in  structured  CTA  interviews  using  the  methods  de¬ 
scribed  in  Clark  et  al. 1  Each  expert  answered  questions  regard¬ 
ing  the  goal  of  the  open  cricothyrotomy  procedure,  the 
necessary  equipment  required,  the  major  steps  of  the  procedure, 
the  indications  and  contraindications  for  performing  the  pro¬ 
cedure,  the  action  and  decision  steps  including  the  conditions 
for  each  alternative  decision,  and  the  standards  required  for 
performing  the  procedure  successfully.  The  transcripts  from 
each  expert  were  used  to  create  a  procedural  CTA  document 
organized  around  the  major  tasks  associated  with  the  procedure 
(Fig.  1),  which  was  then  sent  to  the  respective  surgeons  to 
review.  Once  the  revisions  were  collected,  they  were  compiled 
into  1  gold  standard  CTA  document.  This  document  was  sent 
to  each  expert  for  further  review  and  revision. 

Part  2:  procedural  checklist.  The  gold  standard  CTA  was  ana¬ 
lyzed  to  identify  the  critical  decision  points  associated  with  the 


FIGURE  1.  Major  task  list  for  performing  open  cricothyrotomy 
procedure. 

procedure.  These  decision  points  were  combined  with  the  ac¬ 
tion  steps  and  used  to  generate  a  procedural  checklist.  The 
experts  reviewed  the  task  list  and  procedural  checklist  to  ensure 
conformity  to  the  CTAs.  This  information  served  as  a  frame¬ 
work  for  curriculum  development. 

Part  3:  development  of  eta  curriculum.  The  major  task  list  and 
procedural  checklist  provided  the  curriculum  overview  and 
step-by-step  actions  and  decisions  required  to  perform  an  open 
cricothyrotomy  procedure.  An  instructor  script,  PowerPoint 
presentation  (Microsoft  Corporation,  Redmond,  Washing¬ 
ton),  and  job  aid  for  student  practice  were  developed  from  the 
checklist. 

Subjects  and  Procedure 

Third-year  medical  students  on  the  surgical  clerkship  and  PGY 
2  and  3  surgery  residents  in  the  Department  of  Surgery  at  the 
University  of  Southern  California  (USC)  (n  =  26)  were  as¬ 
signed  randomly  to  either  the  CTA  group  (n  =  12)  or  the 
control  group  (n  =  14).  To  assess  prior  knowledge  and  experi¬ 
ence,  participants  completed  a  6-item  pretest  that  posed  open- 
ended  questions  regarding  actions  and  decisions  required  to 
conduct  the  procedure  given  a  specific  scenario  before  instruc- 
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tion.  Participants  could  score  17  possible  points,  which  were 
based  on  expert  responses.  The  pretest  also  gathered  education 
level  and  previous  experience  performing  an  open  cricothy- 
rotomy.  At  the  end  of  the  instruction  and  practice  session, 
learners  were  given  a  similar  posttest  assessing  their  knowl¬ 
edge  of  the  procedure.  Parallel  forms  of  the  pretest  and  post¬ 
test  instruments  were  developed  using  different  case  scenar¬ 
ios.  To  avoid  pretest  and  post-test  sensitization,  pretest  A 
was  matched  with  posttest  B  and  vice  versa,  respectively.  In 
addition,  learners  completed  a  self-efficacy  rating  scale  after 
the  instruction. 

Participants  in  the  control  group  learned  the  open  cricothy- 
rotomy  procedure  using  the  traditional  method  in  the  Surgical 
Skills  Simulation  and  Education  Center.  This  included  a  didac¬ 
tic  PowerPoint  lecture,  observation  of  a  demonstration  on  an 
inanimate  model,  followed  by  guided  practice  and  feedback. 
The  power  point  presentation  was  comprehensive  and  included 
information  regarding  the  objectives,  indications,  contraindica¬ 
tions,  equipment,  pitfalls,  complications  and  suggested  read¬ 
ings.  In  addition,  the  presentation  included  a  step-by-step  de¬ 
scription  of  the  procedure  with  guiding  pictures.  The  instructor 
for  the  control  group  did  not  participate  in  the  CTA  interviews 
and  did  not  review  any  of  the  CTA  instructional  materials.  The 
instructor  for  the  experimental  group  was  1  of  the  surgeons 
interviewed  to  develop  the  gold  standard  CTA  and  had  re¬ 
viewed  all  the  CTA  materials  before  the  session.  Participants  in 
the  CTA  group  received  the  identical  instruction.  The  only 
difference  between  the  groups  was  the  instructor’s  use  of  the 
CTA-informed  PowerPoint  presentation  and  script,  as  well  as 
the  procedural  job  aid  students  used  during  guided  practice. 
The  instructors  were  matched  regarding  teaching  experience, 
level  of  training,  and  quality  of  teaching  evaluations.  This  study 


was  approved  by  the  USC  Institutional  Review  Board  and  in¬ 
formed  consent  was  obtained. 

Evaluation 

There  were  three  outcome  measures  for  this  study:  ( 1 )  cognitive 
knowledge  of  the  actions  and  decisions  required  to  perform  the 
open  cricothyrotomy  procedure  successfully,  measured  by 
pretests  and  posttests;  (2)  procedural  knowledge  as  demon¬ 
strated  by  the  performance  of  the  actions  and  decisions  steps 
in  the  correct  order,  measured  by  the  procedural  checklist; 
and  (3)  self-efficacy  beliefs  about  performing  the  procedure, 
measured  by  a  self-efficacy  self-appraisal  inventory  adapted 
from  Bandura.13 

Five  expert  surgeons,  who  were  blinded  to  participant 
groups,  evaluated  participants  performing  the  procedure  on  an 
inanimate  model.  Raters  used  a  17-step  procedural  checklist 
and  marked  items  as  “correct,”  “incorrect,”  or  “not  done.”  The 
checklist  also  included  a  question  that  asked  whether  all  the 
items  were  performed  in  the  correct  order  and,  if  not,  which 
items  were  not  performed  sequentially.  Each  step  had  a  value  of 
one  point  except  for  step  2,  which  included  2  additional  sub¬ 
steps  for  a  total  of  19  points  (Table  1). 

The  self-appraisal  scale  for  this  study  included  14  items  in 
which  participants  rated  their  confidence  based  on  a  10- 
point  Likert  scale  with  0  indicating  “Cannot  do  at  all,”  5 
indicating  “Moderately  certain  I  can  do,”  and  10  indicating 
“Certain  I  can  do.”  There  were  140  points  possible  for  the 
self- appraisal  inventory  that  focused  on  the  perceived  capa¬ 
bilities  for  performing  the  action  and  decision  steps  of  the 
task  successfully  (Table  2).  Independent  samples  t  tests  were 
used  for  all  comparisons. 


TABLE  1 .  Procedural  Checklist  for  Open  Cricothyrotomy  Procedure 

Task:  1 

Not  Done  (N)  or 
Incorrect  (1) 

Done 

Correctly 

1 .  Correct  reasons  for  performing  procedure 

N  1 

1 

2.  Prepare  required  equipment 

a.  Test  tracheostomy  tube  for  working  cuff 

N  1 

1 

b.  Pull  inner  cannula  out  of  the  tube  and  put  obturator  into  the  appliance 

N  1 

1 

c.  Assemble  C02  monitor  onto  bag 

N  1 

1 

3.  Correct  patient  position 

N  1 

1 

4.  Prepare  patient 

N  1 

1 

5.  Prepare  self  and  position  self 

N  1 

1 

6.  Stabilize  trachea  with  non-dominant  hand 

N  1 

1 

7.  Identify  incision  location 

N  1 

1 

8.  Make  vertical  incision  (1 .5-2.5  cm) 

N  1 

1 

9.  Retract  area  with  retractor  and  confirm  cricothyroid  location  by  touch 

N  1 

1 

10.  Make  transverse  incision  across  cricothyroid  membrane 

N  1 

1 

1  1 .  Spread  the  cricothyroid  opening  5-10  mm 

N  1 

1 

1  2.  Insert  tube  inside  opening 

N  1 

1 

1  3.  Remove  obturator,  inflate  the  cuff  and  place  inner  cannula  into  tube 

N  1 

1 

14.  Connect  patient  to  C02  monitor  and  check  for  C02  return 

N  1 

1 

15.  Confirm  tube  placement 

N  1 

1 

1  6.  Suction  airway 

N  1 

1 

1  7.  Secure  tube 

N  1 

1 

Total: 
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TABLE  2.  Group  Comparison  for  the  Self  Appraisal  Inventory  Results 

CTA  Group 

Control  Group 

Mean  Score 

Mean  Score 

Self  Appraisal  Inventory  Item 

(N  =  10) 

(N  =  21) 

p-value 

1 .  Recognize  the  indications  for  when  to  perform  the  procedure. 

9.00 

7.33 

0.03 

2.  Recognize  the  contraindications  for  when  not  to  perform  the  procedure. 

9.00 

5.14 

<0.01 

3.  Prepare  yourself  using  universal  safety  precautions. 

10.00 

8.67 

0.01 

4.  Prepare  the  necessary  equipment  to  perform  the  procedure. 

9.60 

9.05 

0.10 

5.  Choose  the  appropriate  tube  for  the  procedure. 

8.60 

8.24 

0.56 

6.  Put  the  patient  in  the  optimal  position. 

9.20 

8.62 

0.19 

7.  Visualize  the  anatomic  landmarks. 

8.80 

8.86 

0.93 

8.  Identify  the  location  to  make  the  incision. 

9.00 

8.86 

0.83 

9.  Make  the  necessary  incisions. 

9.00 

8.24 

0.30 

10.  Place  the  tube  inside  the  opening  correctly. 

9.20 

7.38 

0.01 

1  1 .  Recognize  the  indicators  for  successful  performance. 

9.60 

8.52 

0.01 

12.  Perform  the  procedure  in  an  emergency  situation. 

8.30 

7.24 

0.24 

13.  Perform  the  procedure  in  5  minutes  or  less. 

8.50 

7.29 

0.14 

14.  Perform  the  procedure  without  making  any  major  mistakes. 

8.30 

6.50 

0.03 

Scale:  0  "Cannot  Do  At  All"  to  10  "Certain  I  can  Do. 


RESULTS 

A  data  analysis  revealed  no  difference  between  the  groups  re¬ 
garding  prior  cognitive  knowledge  of  the  procedure  (CTA 
mean  score  5.4,  SD  =  1.68;  control  mean  score  5.5,  SD  = 
1.83;  p  =  0.39).  Therefore,  equivalency  of  the  groups  was 
established.  In  addition,  no  significant  difference  was  found 
between  groups  on  posttest  scores  of  cognitive  knowledge 
(CTAmean  score  18.17,  SD  =  3.61;  control  mean  score  16.71, 
SD  =  2.64;  p  =  0.59).  However,  the  CTA  group  outperformed 
the  control  group  significantly  on  procedural  knowledge  based 
on  a  19-point  procedural  checklist  (CTA  mean  score:  17.75, 
SD  =  2.34;  control  mean  score:  15.14,  SD  =  2.48;  p  =  0.006). 
In  addition,  the  CTA  group  reported  significantly  higher  self- 
efficacy  scores  based  on  a  140-point  self-appraisal  inventory 
(CTA  mean  score:  126.10,  SD  =  16.90;  control:  110.67,  SD  = 
16.8;  p  =  0.029). 

DISCUSSION 

The  results  of  this  study  add  to  the  existing  body  of  research 
demonstrating  that  CTA  is  an  effective  method  to  improve  the 
content  and  instructional  design  of  surgical  skills  curricula.  Pre¬ 
vious  research  demonstrated  that  subjects  who  received  instruc¬ 
tion  based  on  the  results  of  cognitive  task  analysis  interviews 
outperformed  their  colleagues  who  were  not  provided  with  the 
same  instructional  support.8’9,14  It  is  important  to  note  that  all 
these  studies  were  performed  in  a  surgical  skills  center.  How¬ 
ever,  considering  that  fact  that  expert  surgeons  teach  in  the 
clinical  arena,  it  is  reasonable  to  suggest  that  the  benefits  of 
using  CTA  informed  instruction  would  extend  to  the  patient 
care  environment. 

Although  many  medical  education  programs  incorporate 
procedural  checklists  into  surgical  skills  education,  the  strength 
of  the  CTA-informed  checklist  is  that  it  provides  additional 
(and  most  often  expert  omitted)  cognitive  information  about 


when  and  how  the  procedure  should  be  performed.  CTA  pro¬ 
vides  a  method  for  eliciting  knowledge  that  has  accumulated 
through  years  of  experience,  a  framework  for  compiling  that 
information  so  that  it  is  meaningful  to  novices,  and  a  map  for 
tracking  the  performance  of  the  action  and  decision  steps  as 
measurable  points  within  a  procedure. 

The  results  of  this  study  also  demonstrate  the  usefulness  of 
CTA  to  increase  learner  self-efficacy  when  performing  a  medical 
procedure.  These  results  are  promising  because  of  the  paucity  of 
research  in  this  area.  A  search  of  the  medical  literature  did  not 
reveal  any  empiric  evidence  regarding  the  relationship  of  CTA  with 
self-efficacy  in  the  surgical  setting.  Because  increased  self-efficacy 
has  been  linked  to  achievement  in  other  disciplines, 1 0  this  is  an  area 
that  deserves  subsequent  investigation.  Future  studies  should  focus 
on  the  influence  of  CTA  methodology  on  the  relationship  between 
self-efficacy  and  surgical  performance. 

In  addition,  future  research  should  be  conducted  to  focus  on 
examining  the  role  of  CTA  instruction  on  self-efficacy  for  sur¬ 
gical  performance  on  more  complex  surgical  procedures.  The 
open  cricothyrotomy  procedure  is  relatively  straightforward 
and  is  not  performed  very  often,  even  by  expert  surgeons.  How¬ 
ever,  since  the  addition  of  surgical  skills  laboratory  training  for 
surgical  residents,  more  opportunities  are  available  to  examine 
the  effects  of  CTA-informed  instruction  on  trainees.  Because 
CTA  methods  and  CTA-informed  instruction  are  designed  to 
provide  training  for  more  complex  tasks,  a  replication  of  this 
study  targeting  a  surgical  procedure  that  is  complex  and  per¬ 
formed  often  would  be  a  valuable  contribution.  Furthermore, 
future  research  efforts  should  focus  on  the  effects  of  CTA  in¬ 
struction  on  self-efficacy  and  performance  over  a  longer  period. 

Last,  future  research  investigating  the  effectiveness  of  CTA- 
informed  instruction  should  focus  on  patient  outcomes,  such  as 
decreased  complication  rates,  operative  time  of  surgical  proce¬ 
dures,  and  overall  surgical  success.  CTA  research  in  surgical 
education  is  still  in  its  infancy.  As  the  field  develops,  it  will  be 
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important  to  determine  whether  CTA  instruction  has  an  im¬ 
pact  on  long-term  patient  outcomes. 

CTA  has  emerged  in  surgical  education  as  a  promising  re¬ 
source  that  can  be  used  to  extract  the  cognitive  decisions  and 
essential  procedural  steps  of  surgical  instruction  from  experts. 
Information  obtained  from  CTA  interviews  can  be  incorpo¬ 
rated  into  instructional  materials  for  learners  at  all  levels.  Using 
CTA  techniques,  it  is  possible  to  optimize  competency-based 
learning  experiences  with  the  knowledge  and  skills  experts  use 
to  perform  complex  procedures  effectively  and  efficiently.  As  we 
add  to  the  CTA  knowledge  base  and  train  future  CTA  analysts, 
the  goal  of  streamlining  CTA  curriculum  development  for  mul¬ 
tiple  surgical  procedures  is  within  reach. 
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